Illustrations by Ross MacDonald
I’d lost almost $200 million in October. November wasn’t looking any better.
It was 2008, after the Lehman Brothers bankruptcy. Markets were in turmoil. Banks were failing left and right. I worked at a major investment bank, and while I didn’t think the disastrous deal I’d done would cause its collapse, my losses were quickly decimating its commodities profits for the year, along with the potential pay of my more profitable colleagues. I thought my career could be over. I’d already started to feel those other traders and salespeople keeping their distance, as if I’d contracted a disease.
My eyes started to fill from a sudden wash of gratitude and relief that came, I think, from no longer being alone.
I landed in London on the morning of November 4, having flown overnight from New York. I was a derivatives trader, but also the supervisor of the bank’s oil options trading team, about a dozen guys split between Singapore, London, and New York. Until this point I’d managed the deal almost entirely on my own, making the decisions that led to where I ... we ... were now. But after a black cab ride from Heathrow to our Canary Wharf office, I got the guys off the trading floor and into a windowless conference room and confessed: I’d tried everything, but the deal was still hemorrhaging cash. Even worse, it was sprouting new and thorny risks outside my area of expertise. In any case, the world was changing so quickly that my area of expertise was fast becoming obsolete. I pleaded for everyone to pitch in. I said I was open to any ideas.
As I spoke, I noticed that one of the guys had tears welling up in his eyes. I paused for a second, stunned. Then my own eyes started to fill from a sudden wash of gratitude and relief that came, I think, from no longer being alone.
Stress testing is a standard technique derivatives traders use to test how their portfolio will perform in an imagined “worst-case” scenario. The problem is that “worst-case” is subjective, making stress testing as much of an art as a science, and exposing the trader doing the testing to something called the “illusion of control.”
The psychologist Ellen Langer coined the phrase in 1975 to describe “an expectancy of personal success probability inappropriately higher than the objective probability would warrant.” Experimental evidence for the illusion goes back at least to 1965, when one research team found that college-educated employees of AT&T asked to press buttons to illuminate lights had the tendency to believe they had more control than they actually did, even when the lights lit randomly, and even when they used pen and paper to track their results.
In another study, done in 1992, a group of Israeli college students was found to be more willing to bet on dice, and to bet bigger, before they rolled than after, reflecting the belief that they had control over their rolls. Such a preference for prediction over postdiction had been observed before, but this study also found that the preference grew stronger when the students were threatened with an electric shock if they guessed wrong—evidence that stress amplifies the illusion of control.
Langer’s work showed that the illusion is also intensified by “skill cues”: circumstances that make people feel like they’re engaged in acts of skill rather than luck. Such cues include competition, choice, and familiarity with the task at hand. Therefore people will tend to overestimate their prospects in a game of pure chance even more than usual if they face a nervous-looking opponent, or if they pick rather than get assigned a lottery ticket, or if they’re given the chance to familiarize themselves with an apparatus that’s simply spitting out random numbers.
Or, I might add, if they’ve toiled over complicated mathematical equations to back up their decisions.
I thought I knew what I was getting into. I’d helped my bank win the fateful deal by developing a complex option on crude oil, together with a risk management strategy. My equations told me how I could buy and sell simpler financial products over time to approximately offset my daily gains and losses on the option I’d sold.
While I was still executing the trade, I’d stress tested my strategy by imagining oil prices plummeting by 30 percent, option-implied volatilities spiking to record levels, and so on. The “worst-case” loss I derived in September was $30 million, less than a sixth of what I ended up losing in October alone.
And I still had 13 months to go before the option expired.
STANDARDIZED TESTING: The U.S. Federal Reserve has conducted stress tests on the biggest American banks every year since the 2008 financial crisis. Photo by Orhan Cam
I was the archetypical physics Ph.D. hired by Wall Street for my experience modeling the universe with math, based on the belief that the same could be done for markets, particularly for derivatives. I had studied physics because I wanted to make sense of the world—if not to control it, then to at least control my experience of it, to organize it for the comfort that comes with order. When I ended up working for a bank after graduation, I found that I enjoyed it far more than I expected. As in physics, I still had the pleasure of figuring things out, and I still had a chance to get to the bottom of things, even if only in one small part of the world.
By 2008, I’d spent over a decade developing derivatives and trading strategies, most of them successful. And I was successful too, not for any ability to forecast the future, but simply for being careful and logical and systematic. Basically Wall Street rewarded me for doing the sorts of things I’d hoped to do in physics; the only difference was that I now applied probability theory to markets rather than quantum mechanics.
But until 2008, I’d never really experienced the extreme tail of a probability distribution firsthand. And that experience disabused me of more than one illusion.
The derivative I’d sold was an option, but it worked just like an insurance policy. The buyer was the government of Mexico, the sixth largest exporter of crude oil in the world at the time, and a country dependent on those exports for about a third of its federal budget. A crash in crude’s price could cause deficits or the need for drastic cuts to social programs, neither of which was particularly palatable to the country’s politicians. Traditionally, Mexico had used derivatives to lock in a minimum sale price for the crude it would sell the following year. In 2008, they covered a fraction of their 2009 exposure with me.
I got wind of the leak in an email from our head salesperson, who added the comment, "This press coverage is going to kill us."
The option they bought from me was complex, or at least unusual, in that it was written on Maya crude, a heavy and sour grade comprising nearly 90 percent of Mexico’s production. For years, Mexico had hedged using options on West Texas Intermediate (WTI), the United States crude most commonly quoted in the news as “the price of oil,” and the most heavily traded commodity in the world, thanks to the extensive derivative market that trades on it. For Mexico, buying WTI options was convenient and cheap, like buying a suit off the rack. But it also exposed the country to the risk that Maya might underperform WTI in the event of a crash, in which case its off-the-rack insurance might fail to cover its made-to-order losses.
Enter my bank and I, offering Mexico a custom suit in the form of an option on Maya. It cost more, but the strategists in Mexico’s finance ministry liked it enough to pay the premium, and to let my bank participate for the first time in its annual hedging program. This was a big deal for my bank because it was vying to be the top commodities bank in the world, and Mexico’s program was the largest and most prestigious of its kind. It was a big deal for me too, given its potential impact on my prospects for promotion and a bonus.
But there was a reason no other bank had sold Maya options to Mexico before, and it was the same reason I found myself contemplating a career change in November of 2008: Unlike WTI, there was no derivatives market on Maya, and therefore no opportunity for me to hedge my risk using standard derivative techniques. I had to improvise.
As with my stress testing, I depended on historical data—how Maya’s value had changed in the past relative to that of other oil and refined products—to drive my hedging strategy, which involved proxying Maya with a basket of other hydrocarbons, including fuel oil. Fuel is a refined product like gasoline or diesel, but of lower quality and used to power ships. Very roughly, you could think of my proxy for Maya as a blend of WTI and fuel. The key was that, unlike with Maya, I could trade derivatives on fuel and WTI in the interbank market. The problem was what happened to that market as oil prices crashed.
As prices sunk lower and lower, and it became more and more likely that Mexico’s insurance would pay off—so that I would effectively be responsible for buying the country’s 2009 Maya at the price I’d guaranteed it—my strategy required me to sell more and more fuel to hedge my increasing exposure.
And therein lay the rub.
In markets, at least, size matters. I said earlier that I’d insured a fraction of Mexico’s exports. What I didn’t say was that that fraction was two-thirds: two-thirds of the world’s sixth largest oil exporter’s product, 220 million barrels, or around $17 billion worth, enough to fill 100 supertankers and then some. That meant I had to sell a lot of fuel. More, as I quickly and painfully found out, than the market could bear.
THE BIG DEAL: Our deal with Mexico insured two-thirds of their oil exports, enough to fill more than 100 supertankers. Photo by Harald Sund/Getty
Through October, I couldn’t sell fuel fast enough to keep up with my exploding exposure. Making matters worse, fuel’s price fell far faster, relative to other oil products, than it ever had before. I lost money on all manner of risks in those weeks, but about half of the damage came from fuel.
And, adding insult to injury, part of the reason for fuel’s exceptional weakness was my own feverish selling. The fuel market is small and the deal was ginormous, and so my sales flooded the market with both fuel and information about my position, weakening my bargaining power and making it harder to sell as the days went by. Hitting a market’s ceiling like this was something that none of my methodologies accounted for.
Nor did they account for other information leaks, beyond fuel. The same week I landed in London, Mexico’s finance minister broke with convention and—trying to reassure markets about his country’s finances—spilled the beans about the deal to a reporter. I got wind of the leak in an email from our head salesperson, who added the comment, “This press coverage is going to kill us.”
Yes.
Success in “normal” circumstances says little about your chances in crisis, since in crisis the rules change. But it sure can make you feel confident—which is how I felt after the successes I’d had before 2008.
A 2014 study by Helena Matute, the director of the experimental psychology lab at the Unversidad de Duesto in Bilbao, Spain, asked 50 pairs of participants to administer fictional medicine to fictional patients suffering from a fictional disease. Half had 30 doses of the medicine to disburse among 100 patients, and the other half had 70. Eighty percent of the patients recovered, regardless of whether or not they were given medicine.
Slightly less than half of the participants giving 30 doses believed that the medicine was effective, but more than two-thirds of the 70-dose crowd believed they’d played a role in patients’ recovery. The experience of repetition, and greater personal involvement, strengthens the illusion of agency even with no causality. Success with some causality, like mine, surely must strengthen it even more.
People would avoid cars, planes, marriages ... everything really.
This happens on the institutional level too, Exhibit A being the Federal Reserve, which has conducted stress tests on the biggest banks every year since the crisis. The Fed’s goal is the same as mine was in September 2008: to calculate how bad things might get. In the Fed’s case, that means testing what might happen to the banks’ capital (assets minus debts) in a crisis when assets depreciate. Negative capital is called insolvency. For a bank, insolvency is a fast track to bankruptcy. In “supervisory” stress testing, regulators like the Fed—this is being done in Europe too—posit one or more worst-case scenarios and compute the consequences for capital, similar to how I tried to calculate how much I’d lose in a crash.
The Fed’s first test in 2009 was an unmitigated success, widely credited with ending the most acute stage of the crisis. The test was based on a scenario in which the market tanked even further than it already had and a deep recession followed. The Fed concluded that several big banks would suffer capital shortfalls as a result, Bank of America leading the list with a deficit of $33 billion. The Treasury Department then topped off the needy banks with cash from its Troubled Asset Relief Program, reassuring markets and kick-starting a recovery.
Now, the Fed stress tests the big banks every year, partly because the chief piece of post-crisis financial regulation in the U.S., the 2010 Dodd-Frank Act, requires it. Banks have long operated under regulatory capital limits, such as minimum ratios of capital to assets. What’s new since the crisis is that those minima have increased and that stress testing is used to regulate banks’ capital plans—limiting share buybacks and dividend payments, for example—to try to guarantee that they’ll have sufficient capital no matter what.
The Fed’s test in 2015 concluded that the 31 largest bank holding companies in the U.S., which account for more than 80 percent of the country’s banking assets, would lose a grand total of $490 billion in its worst-case scenario. And yet, for the first time in the tests’ history, not a single bank failed the test by having its capital to asset ratio fall below the Fed’s 5 percent hurdle. The implication was that the banks were finally under control.
Or was it just an illusion of control?
I took a look at the Fed’s stress test scenario1 myself recently, on the bank’s website. My first reaction was that it seemed almost farcically fashioned to “fight the last war”: The Dow drops by about half, U.S. GDP dips around 5 percent, unemployment spikes to 10 percent—basically it’s the aftermath of 2008 all over again. It doesn’t account for other potential calamities, like a breakup of the Euro, for example, or an emerging market crisis, or hyperinflation, or shock-induced feedback effects like the ones I faced with fuel.
There’s also the inconvenient fact that both Fannie Mae and Freddie Mac were stress tested regularly by their regulator, and declared well capitalized—right up until they failed in 2008. The same was true for Iceland’s banks.
I’m not alone in my skepticism. The former Fed economist Til Schuermann, who had a hand in designing the Fed’s tests, thinks that they actually add risk to the system, rather than reduce it, a position he outlined in a 2013 Wall Street Journal op-ed. “The danger is that the financial system and its regulators are moving to a narrow risk-model gene pool that is highly vulnerable to the next financial virus,” he wrote. “By discouraging innovation in risk models, we risk sowing the seeds of our next systemic crisis.”
In other words, the Fed, by centralizing stress testing around its own approach, is incentivizing banks to follow suit, which may push them to accumulate similar exposures to one another and to manage them in similar ways, resulting in decreased diversification and increased risk. This is a question raised by particularly prescriptive rules like the Fed’s 5 percent hurdle, which are simple to monitor but may be just as simple to game.
What’s more, the Fed’s own incentives may support an illusion of control.
“They’re grading their own papers and they always pass,” said Kevin Dowd recently on a Cato Institute podcast. Dowd, another Fed stress test skeptic, is a professor of finance and economics at Durham University in England. “A central bank stress test can never be credible because of the incentives built in to get a pass result,” he added.
Dowd reasons that, given the Fed’s mandate to keep the financial system safe, it’s motivated to find that it is safe, similar to the way that I was incentivized to get stress testing results that supported doing my potentially mega-profitable deal.
You don’t argue when someone’s handing you a big check, but I wondered what the hell he was thinking.
I got Dowd on the phone recently and asked him what the Fed could do to improve its tests.
“First off, don’t do them,” was his response. So what model should the Fed use instead?
“I would not use any model at all. I do not believe financial risk modeling works,” he replied.
“It takes a model to beat a model,” I challenged, plagiarizing a phrase I’d learned from another professor of finance when I started in banking and which became something of a mantra of mine thereafter.
Dowd chuckled, but was unmoved. “I would always go for history over a model,” he replied, before explaining that he’d look to what capital levels were in the 19th century, when banks were private companies and less subject to government regulation.
But even history requires a model to interpret it. Riccardo Rebonato, global head of interest rates and FX at Pimco, and author of one of the very few books dedicated to the art of stress testing, made this point when I called him, by asking me to imagine that a Martian had landed on Earth.
Suppose, he said, that the extraterrestrial and I both witnessed the same event: another human being pulling a penny from his pocket, flipping it four times, and having heads come up each time.
I, Rebonato guessed, wouldn’t think much of this coincidence, because of my past experience with coins. Four heads in a row is not an especially rare event. So I’d probably still guess the odds of heads coming up a fifth time were 50-50.
But, he proposed, “The Martian knows absolutely nothing about coins. For all he knows coins are devices for producing heads when you flip them.” Therefore the Martian would be more likely to bet on heads against me.
“The point of the story,” Rebonato continued, “is that you always come to data with a structural model behind”—meaning some preconception of causes and effects, and therefore some prejudice for how to interpret the data.
I understood immediately, having spent countless hours over many years struggling to extract probabilities and other forward-looking metrics from historical data. The biggest challenge was always that the answers depended so much on which timeframe you focused on. The past month? Year? Five years? And if you want to use data to estimate the probability of a very rare event, then you need a lot of data—the more rare, the more data—which means taking data from further back in the past, and the further back you go, the less relevant it probably is.
Bob Henderson.
But what’s the alternative to estimating probabilities? If we can’t say anything about what the probabilities are, then we really are forced to worry about the absolute worst possible scenario. A trader would never sell options or short stocks, since his losses could be infinite. People would avoid cars, planes, marriages ... everything really, for fear of the worst possible result. We’d all be paralyzed.
Which is why Rebonato told me that professed ignorance of probabilities is “a form of tyranny that’s an excuse for inaction.” And why his favored approach to stress testing (a technique called Bayesian nets) doesn’t involve trying to derive probabilities at all, but rather asks the user to just “produce your best-informed guess.”
The most stressful part of my 2008 experience taught me how to guess.
The day after Lehman fell I lost $20 million, and the day after that $30 million—enough in two days to wipe out all the profits I’d made the previous year. (And that had been a pretty good year.)
But worse was that I felt trapped. My models showed I was destined to lose far more money in the coming weeks, no matter what I did. All roads seemed to lead to an unavoidable abyss. I could practically feel that hot hole breathing under my desk. I actually got dizzy, and lost my ability to think. When my boss stopped by to warn me that Goldman Sachs and Morgan Stanley looked likely to fall next, he seemed almost amused when he told me that I looked green.
I stumbled home early that day, mentally incapacitated for the first time in my career.
But stress had other effects too.
Psychologists say that the illusion of control can be adaptive, in the sense that it encourages a focus on problem-solving behaviors as opposed to emotional response. So once I got past flight and started to fight, I may have had a dose of delusional thinking to thank for the guesses I made next.
The Sunday after Lehman fell, pacing my empty trading floor, I realized once and for all that my models and reports could no longer tell me what to do. The one unmistakable fact was that my risks would increase if oil continued its decline. I decided that when I came in on Monday, I’d place a big bet that WTI would do just that.
And on a Saturday morning bike ride up the Hudson, it occurred to me that Mexico might be willing to restructure its deal—selling us back the option it owned, and buying a new one—in a way that would lock in billions of profits for the country, while giving me a much needed windfall too. I dropped my bike in a bush and texted our salesperson about the idea.
There were many other decisions and guesses, some made alone, others with help from my team, and still others made by my boss. All were guesswork, none could I have anticipated in stress testing, and all involved abandoning my original strategy along with the illusion of control it gave me.
In their 2003 paper “Trading on Illusions,” the organizational-behavior researcher Mark Fenton-O’Creevy and his colleagues argued that the stress, competition, and choice involved in trading financial instruments naturally give rise to illusions of control, something my own experience affirmed. They also found that those illusions are inversely related to trader performance. They even got it down to a number: “An increase in illusion of control of one standard deviation is associated with a decrease in annual remuneration of £58,000 [$87,500].”
Having relinquished at least some of my illusions, I couldn’t be sure that what happened next wasn’t just dumb luck.
First, Mexico bought into my idea of a restructure, executed the trade, and allowed me to make enough on it to fill a good fraction of the hole left by another $200 million of losses I suffered that month on my original risk.
Then WTI continued its decline, hitting bottom in December at $33 and change. I made a killing on my short position, unwound it, and started making money on other bets, on fuel and volatility, for example. December went so well that I recouped most of my losses by the end of the year.
Then, in 2009, persistently low oil prices caused me to accumulate a massive position in Maya, all 220 million barrels, which strengthened significantly relative to my hedges, something that my original model said should happen, but which could have been more fluke than foresight. In any case, things went so well during the first quarter that one executive visiting from Singapore marveled at how I was “printing money faster than Obama.”
The deal matured in December of 2009, with happy repercussions all around. Mexico netted over $6 billion and kept its budget whole. Futures and Options World magazine bestowed its award for “Most Innovative or Creative Use of Derivatives” on the country and presented it to the minister of finance at a press conference in New York that some of my colleagues and I attended. The minister was lauded in the press for his prescience, and was ultimately made, as I heard was his ambition, the head of Mexico’s central bank.
I got promoted too, paid a big bonus, and was told that I’d presided over the most profitable deal in my bank’s history.
In the February 2010 meeting in which my boss gave me my number, he said that to his mind I’d earned it more for the part I’d played in winning the deal in the first place than by all the work I’d done after.
You don’t argue when someone’s handing you a big check, but I wondered what the hell he was thinking. To me, everything important had come after. Only by managing that monstrosity through the crisis did I come to fully appreciate how unlike physics markets are, how crucial to outcomes other people and luck are, and how, no matter how hard I worked or how much I prepared, there would always be some things I couldn’t completely fathom.
Bob Henderson studied physics, worked on Wall Street, and is now an independent writer focused on science and finance.